Minimum Entropy Estimation in Semi-Parametric Models: a candidate for adaptive estimation?

In regression problems with errors having an unknown density f , least squares or robust M -estimation is the usual alternative to maximum likelihood, with the loss of asymptotic efficiency as a consequence. The search for efficiency in the absence of knowledge of f (adaptive estimation) has motivated a large amount of work, see in particular (Stein, 1956; Stone, 1975; Bickel, 1982) and the review paper (Manski, 1984). The present paper continues the work initiated in (Pronzato and Thierry, 2001a,b). The estimator is obtained by minimizing (an estimate of) the entropy of the symmetrized residuals. Connections and differences with previous work are indicated. The focus is mainly on the location model but we show how the results can be extended to nonlinear regression problems, in particular when the design consists of replications of a fixed design. Numerical results illustrate that asymptotic efficiency is not necessarily in conflict with robustness.